Skip to content

split/gfx scene#2077

Open
jcelerier wants to merge 75 commits into
masterfrom
split/gfx-scene
Open

split/gfx scene#2077
jcelerier wants to merge 75 commits into
masterfrom
split/gfx-scene

Conversation

@jcelerier

Copy link
Copy Markdown
Member
  • isf: TEX_DIMENSIONS_3D / IMG_SIZE_3D aliases for 3D samplers
  • isf: point3d_input AS_COLOR flag for color-swatch display
  • isf: parse-time warning on unknown csf_image_input FORMAT
  • isf: parser rework — top-level descriptors, uniform inputs, geometry AUXILIARY, vertex inputs
  • 3rdparty: add OffsetAllocator submodule
  • gfx: add AssetTable + TextureLoader for shared decoded-asset cache
  • gfx: add GpuResourceRegistry — slab-allocated GPU arenas backed by OffsetAllocator
  • gfx: add CameraMath / GpuTiming / VertexFallback helpers
  • gfx: add IsfBindingsBuilder + PipelineStateHelpers
  • gfx: add SceneGPUState — FlatScene + scene packer
  • gfx: add OffscreenDevice + RhiPreviewWidget + Metal buffer-copy backend
  • gfx: extend ISF / SimpleRenderedISF nodes for new ISF features
  • csf: rework for scene-aware compute pipeline
  • gfx: rework raw raster pipeline (MRT, AUXILIARY, EXECUTION_MODEL) and VSA
  • gfx: refresh shared graph plumbing for incremental scene pipeline
  • gfx: add ScenePreprocessorNode — scene_spec to flat draws + arena uploads
  • gfx: add SceneFilterNode + FlattenedSceneFilterNode + MergeGeometriesNode
  • avnd: split CpuFilter / CpuAnalysis lifecycle into init / initState / release
  • avnd: add scene_port concept
  • avnd: split GpuNode lifecycle and add scene_port storage helpers
  • gfx: extend ShaderProgram for new ISF features
  • gfx: refresh Filter / GeometryFilter / TexturePort for new pipeline
  • gfx: refresh WindowDevice / Spout / Syphon / WindowCapture
  • gfx: refresh GfxContext + window / screen / multiwindow nodes for incremental scene rebuild
  • js: gpu node lifecycle rework with deterministic teardown
  • wip: many bugfixes across the board
  • build: more build fixes
  • ci: fixes for older qt versions
  • rhi: populate caps
  • ci: try windows fixes
  • gfx: restore PhongNode, still used by score-vfx-template
  • gfx: keep TextureRenderTarget compatible with addons' aggregate init
  • gfx: do not include mmsystem.h in CommonUBOs.hpp
  • gfx: fix build with Qt < 6.9
  • gfx: fix designator order after TextureRenderTarget reorder
  • gfx: fix build with Qt 6.4
  • gfx: more Qt 6.4 compatibility guards

@jcelerier jcelerier force-pushed the split/gfx-scene branch 2 times, most recently from 18d8ecf to baed148 Compare June 18, 2026 16:29
jcelerier added a commit that referenced this pull request Jun 19, 2026
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
jcelerier added a commit that referenced this pull request Jun 19, 2026
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
@jcelerier jcelerier force-pushed the split/gfx-scene branch 2 times, most recently from db9b8dc to 1c1f6c8 Compare June 25, 2026 19:16
jcelerier added a commit that referenced this pull request Jun 25, 2026
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
jcelerier added a commit that referenced this pull request Jun 26, 2026
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
jcelerier added a commit that referenced this pull request Jun 26, 2026
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
jcelerier added 19 commits July 6, 2026 21:02
…AUXILIARY, vertex inputs

Top-level descriptors (PIPELINE_STATE, MULTIVIEW, EXECUTION_MODEL, EXTENSIONS,
CLIP_DISTANCES / CULL_DISTANCES, DEPTH_LAYOUT). New input kinds: uniform_input
(UBO INPUTS), per-input sampler_config (WRAP / FILTER / COMPARE / MIPS),
audio_sampler_config. Storage / image / cubemap input extensions: PERSISTENT,
IS_ARRAY, GENERATE_MIPS, cube/3D textures. Geometry input AUXILIARY: storage +
sampled, persistent, is_uniform, depth companion, INDIRECT block. Per-pass:
LAYER, Z, FORMAT, PIPELINE_STATE override. OUTPUTS: LAYERS, DEPTH, FORMAT,
SAMPLES, CUBEMAP, GENERATE_MIPS, WIDTH/HEIGHT expressions. Vertex inputs:
REQUIRED, DEFAULT, SEMANTIC, INTERPOLATION (with auto-flat for int/bool
varyings).

Bug fixes: mat3/mat4 attribute slot count, TYPES struct emission in BOTH
stages, drop incorrect std430 padding heuristic that was breaking RawLight
stride.
jcelerier and others added 4 commits July 6, 2026 21:05
- BackgroundNode.hpp: include <score/application/GUIApplicationContext.hpp>
  (called score::GUIAppContext() via a broken transitive include; non-unity only)
- CommonUBOs.hpp: drop the <windows.h> + '#undef near/far' block (matches master)
  and rename the colliding UBO fields near/far -> znear/zfar. The '#undef far'
  leaked into downstream Windows headers, breaking mmeapi.h (CameraDevice.win32)
  and combaseapi.h (MSVC).
- CameraDevice.win32.cpp: clang-format off/on around the order-sensitive Windows
  includes so SortIncludes can't reorder and re-break them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UJoeDPZWGZMVPe65Np4f2F
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
jcelerier and others added 25 commits July 7, 2026 09:35
PassMap changed from pair<Edge*, Pipeline> to pair<Edge*, Pass>; addons
like score-vfx-template still emplace a Pipeline per edge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
- RenderList::onEdgeRemoved: rebind surviving sink samplers to the empty
  texture before releasing their input render target; the incremental
  path dropped master's implicit rebind-everything guarantee (UAF on
  live cable disconnect).
- MultiWindowNode::setRenderSize: recreate the offscreen target before
  the render-list rebuild again (the synchronous rebuild path compiled
  pipelines against pointers freed right after); marshal the OSC
  rendersize callback to the Qt thread like the other callbacks.
- Crousti GpuNode/GpuComputeNode: recreate the SRB after the
  srb_touched destroy path; it was only created on first init, so any
  later resource reallocation bound a null native descriptor set.
- GaussianSplat: null the batch after the preprocess and depth-key
  beginComputePass — SH-preprocess frames double-submitted it.
- RawRaster: build the MRT pipeline against the per-iteration render
  pass in execution-model modes (PerMip/PerCubeFace/PerLayer color) —
  the N-attachment multisampled MRT descriptor is incompatible with the
  1-color 1-sample iteration RTs; fix depth sample-count mismatch in
  the no-declared-depth MRT branch.
- ISF: key persistent-SSBO ping-pong and MRT render-once guards on the
  RenderList frame counter — update() runs once per sink, so the bool
  guards double-swapped/double-rendered with more than one output edge;
  SimpleRenderedISF now also re-applies swapped bindings to every SRB.
- Soft-fail instead of throwing on the render thread: batch-pool
  exhaustion in RenderList::render, empty m_passes after a failed
  pipeline build in RenderedISFNode::update.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
- ShaderProgram: removeIncludesInComments hung on backslash-escaped
  quotes (loop never advanced).
- Window::exposeEvent: restore the null-swapchain self-heal; teardown
  runs on the render thread with plain writes, so the GUI thread can
  observe the inconsistent (flag, pointer) pair mid-teardown.
- ModelCameraUBO: pack modelNormal as three vec4-aligned columns as
  std140 mat3 requires; 9 contiguous floats garbled columns 1-2.
- GenericNodeRenderer: key the pipeline cache on the renderpass
  serializedFormat() (QRhi's documented compatibility key) instead of
  the rp-desc pointer, which could alias a freed descriptor.
- ISF: seed the MULTIVIEW viewProjection UBO with identities; it was
  never written, so multiview shaders collapsed all vertices to zero.
- WindowCapture (pipewire): fix ARGB8888/XBGR8888 fourccs (were the
  invalid 'RA24'/'BX24'); double-buffer the CPU frame so grab()'s
  pointer survives the capture thread's next resize; publish a dup of
  the DMA-BUF fd so imports don't race the requeued buffer's teardown.
- Graph::reconcileAllRenderLists: don't dereference a null batch when
  the update-batch pool is exhausted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
- ScenePreprocessorNode: cascade_split_distances slot k must hold the
  FAR plane of cascade k (boundary k+1); copying boundary k shifted
  every split by one slice and dropped the real far distance at 8
  cascades.
- CustomMesh: skip sub-meshes whose bindings/attributes differ from
  meshes[0] — the only layout the pipeline is built for — instead of
  fetching them through the wrong strides.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
- reconcileAllRenderLists: on batch-pool exhaustion, skip renderer
  CREATION but still run step 7 — the earlier  left freed
  renderer pointers (deleted in step 3) live in rl->renderers → UAF.
- ModelDisplay (gfx-scene copy): pack modelNormal as std140 columns;
  toGL deduced N=12 from the enlarged array and memcpy'd 48 bytes from
  a 36-byte QMatrix3x3 (OOB read) after CommonUBOs grew the field.
- GenericNodeRenderer pipeline cache: fold sample count + multiview
  into the key. serializedFormat() omits sample count on Metal/D3D
  (empty on GL), so two out-edges at different sample counts shared a
  wrongly-multisampled pipeline.
- RenderedISFNode::runInitialPasses: guard empty m_passes before [0].
- RenderList::render: guard the two remaining unchecked
  nextResourceUpdateBatch() sites (warn + skip frame).
- WindowCapture: grab() hands the consumer its own dup of the DMA-BUF
  fd (consumer closes it); the shared dup was closed by the capture
  thread mid-import.
- VSA DataStreamReader: guard documentContext() like the Filter/ISF
  siblings — serializing outside a document asserted/crashed.
- RenderedCSFNode: acquire a batch before generateMips; an image-only
  CSF reached it with res nulled by the preceding compute pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
R3-N4: the API-specific cull mode was set BEFORE m_mesh->preparePipeline(),
which unconditionally re-sets cullMode/frontFace (DummyMesh keeps the base
CullMode::None). The cull state was therefore dead and every triangle-mode
VSA shader rendered backfaces on all backends. Move the cull/front-face block
to after preparePipeline() so it wins.

R3-N5: the initPass failure paths (ps->create() failing, or any throw out of
makeShaders / SCORE_ASSERT) leaked the 6 background-pass GPU objects via an
empty catch(...). Add a releaseBackground() helper and release ps/srb/pubo +
background objects on both failure exits. bg_tri is left alone (its buffers
are cached/owned by RenderList::m_vertexBuffers).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
…en binding

R3-N1: RenderedCSFNode had no once-per-frame guard, so a CSF feeding >=2 sinks
re-ran every compute pass and double-swapped feedback SSBOs / persistent images
once per outgoing edge (simulation advancing at N x). Add int64 m_lastRunFrame
keyed on renderer.frame (mirrors SimpleRenderedISFNode::m_lastMRTRenderFrame),
checked at the top of runInitialPasses and reset in releaseState().

R3-N3: registerCommonExpressionVariables under-reserved the exprtk symbol
vector (2 doubles/image, nothing for the per-buffer $COUNT/$BYTESIZE pair).
ossia::math_expression::add_constant stores a double& into that vector, so an
overrun reallocated and dangled every prior reference. Add
expressionSymbolReserveCount() (a proper upper bound) and use it at all three
reserve sites.

R3-N2: initState's hand-rolled port/outlet counters drifted from ISFNode's
actual port creation (ISFNode.cpp isf_input_port_vis):
 - a write storage_input with a flexible-array layout synthesizes a long_input
   sizing inlet — now skipped (was only done for read_only);
 - $USER in INDIRECT.COUNT was phantom-counted as an input port in BOTH
   initState and buildComputeSrbBindings — ISFNode creates no port for it, so
   both increments are removed;
 - a writable AUXILIARY set has_output=true and over-advanced outlet_index for
   a Geometry outlet that does not exist — advance outlet_index on the canonical
   rule (empty attrs or a writable ATTRIBUTE) instead, and bounds-guard the
   outlets[outlet_index] writes.

R3-N8 (codegen): the material Params-UBO binding++ ran even when the block was
dropped (k==0, e.g. a no-parameter write-image/fixed-storage generator),
shifting every later GLSL binding +1 vs the runtime SRB (which gates binding 2
on m_materialSize>0). Move binding++ inside if(k>0).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
…expr reserve

R3-N7: m_perInvocationSRBs (PER_LAYER / PER_MIP / MANUAL COUNT>1 clones of the
main SRB) were not re-patched when input-port textures or geometry-published
buffers/images were swapped — the persistent-aux and aux-texture paths already
mirrored into the pool, but updateInputTexture and the two
bindUpstream*FromGeometry refresh loops did not. QRhi generation-tracking only
covers rebuilding the SAME object; these swap to different QRhiTexture*/
QRhiBuffer*, so invocations 1..N-1 kept stale pointers -> UAF/garbage on every
layer/mip past the first when upstream reallocated. Mirror all three paths into
the per-invocation pool.

R3-N6: createDepthOnlyRenderTarget attached a 1x1 dummy color attachment 0. The
Vulkan backend derives the framebuffer / render area from color attachment 0
whenever colorAttCount>0 (qrhivulkan.cpp:8290-8293; the depth-size fallback only
fires at colorAttCount==0), so depth written beyond pixel (0,0) was undefined —
shadow_cascades / PER_LAYER depth then copyTexture()'d a full target of garbage.
Size the dummy color to the depth extent (both overloads). MRT paths use a
separate createRenderTarget and are unaffected.

R3-N3 (RawRaster twin): resolveIntExpression reserved a fixed 16 doubles for the
exprtk symbol vector; add_constant stores double& into it, so images+buffers
past 16 reallocated and dangled the references. Reserve a proper upper bound.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
…Spout16 doc

R3-N17: FrameQueue seek-discard was a TOCTOU. VideoDecoder set_discard_frame(F)
then enqueue(F) were non-atomic, so the GFX thread's discard_and_dequeue* could
observe the marker for F, drain the queue and return F as current while the
decoder still owned/enqueued it -> double ownership / UAF of pixels a live
zero-copy upload referenced. Enqueue F BEFORE publishing the marker, and make
discard_and_dequeue / discard_and_dequeue_one return the marker frame only when
it is actually found in the queue (otherwise fall through instead of returning a
frame we don't own).

R3-N18: HWTransferDecoder mid-stream sw-format change reset m_delegate after only
clearing its samplers vector, running the empty ~GPUVideoDecoder and leaking the
old delegate's QRhiTextures/samplers every change. Call the base
m_delegate->release(r) first, then reset. Note added that the renderer's SRBs
built from the old samplers are now stale and must be rebuilt at node level (the
decoder has no handle on them).

R3-N19: Spout R16G16B16A16_UNORM senders are imported as RGBA16F/SFLOAT (the only
4x16 QRhi format); reinterpreting UNORM bits as half-float makes bright values
(>= 0x7C01, 1.0 = 0xFFFF) sample as NaN/Inf that propagate through blends. A
post-sample pass cannot recover NaN and the correct fix needs a format-converting
import (vkCmdBlitImage / D3D shader copy) on a Windows-only path; documented the
hazard precisely at both format-mapping sites. NOT fixed — see final note.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
An ISF shader declaring BOTH multiple color OUTPUTS (MRT) AND a
fragment-visible storage input (SSBO, e.g. PERSISTENT read_write)
crashed on Vulkan and left the 2nd attachment blank on OpenGL.

SimpleRenderedISFNode::initMRTPass built its pipeline's extra bindings
via buildExtraBindings(m_storage) but — unlike the non-MRT initPass —
never called ensureStorageResources()/bindUpstreamBuffers() first. So
the storage buffers were never allocated; buildExtraBindings skips any
entry whose GPU buffer is still null (IsfBindingsBuilder.cpp:
`if(!e.buffer || e.binding < 0) continue`), and the fragment storage
buffers the codegen declares (SPIR-V Set 0, bindings 3/4 for a
persistent read_write SSBO) were omitted from the MRT SRB. On Vulkan
the SRB *is* the pipeline layout (qrhivulkan derives VkPipelineLayout
from it), so vkCreateGraphicsPipelines hit VUID-...-layout-07988 and
the draw SIGSEGV'd on the missing descriptor; on GL the 2nd attachment
rendered blank.

Allocate the storage resources and borrow upstream buffers before
buildExtraBindings in initMRTPass, mirroring initPass, so the MRT SRB
carries the same storage bindings the shader uses. ensureStorageResources
is idempotent (guarded on e.buffer), so it runs once when the shared MRT
target is first built.

Fixes the isf-mrt-persistent-ssbo L3 finding.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
…ffer

A multipass ISF shader that also declared a storage/persistent SSBO
rendered an all-black final pass on both backends, though plain
multipass and single-pass persistent SSBO both worked.

A write/read_write storage_input declares a Types::Buffer OUTPUT port,
and ISFNode walks desc.inputs (appending those buffer output ports)
BEFORE it appends the implicit color output. So for a multipass shader
that also uses a storage buffer, output[0] is the buffer port and the
color output lands at output[1]. RenderedISFNode::init hardcoded
n.output[0]->edges, so it created passes for the (edge-less) buffer
port and none for the color output; runRenderPass then found no pass
for the sink's edge and the final pass never reached the sink.

Iterate every output port and restrict to Types::Image (mirroring
SimpleRenderedISFNode::init, which already did this and is why the
single-pass persistent path was unaffected). Non-storage multipass is
unchanged: its sole output port is the Image color port.

Fixes the isf-multipass-storage-rw and isf-multipass-persistent-ssbo
L3 findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
SimpleRenderedISFNode::textureForOutput() mapped an OUTPUT Port to its MRT
color/depth attachment by indexing descriptor().outputs with the port's raw
index in n.output. But n.output can INTERLEAVE non-image output ports among
the image ones: a write/read_write storage_input pushes a Types::Buffer
output port (and a writable geometry_input a Types::Geometry one) during
ISFNode's desc.inputs walk (ISFNode.cpp:215), BEFORE ISFNode appends one
Types::Image port per desc.outputs entry (ISFNode.cpp:354). Meanwhile
initMRTPass builds the attachments by iterating descriptor().outputs, which
lists only the image/depth outputs.

So for an MRT shader that also declares a storage buffer (e.g.
isf-mrt-persistent-ssbo.fs: n.output = [Buffer, outA, outB] vs
descriptor().outputs = [outA, outB]), every color output shifted by one: the
1st color (outA) sampled the 2nd attachment and the 2nd color (outB) ran past
outputs.size() and returned nullptr -> attachment 2 read back black.

Fix: walk n.output but advance the descriptor index only for Types::Image
ports, skipping Buffer/Geometry, so the image ports line up 1:1 with
descriptor().outputs and the attachment order initMRTPass produced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
A front-facing VSA triangle (vsa-triangle.vs) drew on OpenGL but was culled
entirely on Vulkan; reversing the winding flipped which backend culled. No
single winding rendered on both, so the per-backend cull (Front on GL, Back
on Vulkan) + FrontFace CW could not make one triangle visible cross-backend.

Root cause: face culling is decided from the triangle's window-space winding
sign, which QRhi does NOT normalise across backends (GL passes frontFace
straight to glFrontFace; Vulkan uses a positive-height, Y-down viewport).
Cross-backend winding consistency holds ONLY for shaders that follow the QRhi
convention -- multiply gl_Position by clipSpaceCorrMatrix and do NOT flip Y on
SPIRV/Vulkan -- which is exactly what the consistent paths do (ISF blit_vs,
RenderedRawRasterPipelineNode), then culling with a single CullMode::Back.

VSA does the opposite (libisf isf.cpp:5620): it skips clipSpaceCorrMatrix and
manually does gl_Position.y = -y on SPIRV/HLSL/MSL. That keeps the rendered
image ORIENTATION consistent across backends but INVERTS the window-space
winding sign on Vulkan relative to GL, so for any winding exactly one backend
keeps the face and the other culls it -- no per-backend cull can fix it.

VSA art is 2-D procedural geometry driven by gl_VertexIndex; front/back face
is not meaningful for it. Draw both faces (CullMode::None): visible AND
identical on every backend. Points/line VSA modes are unaffected (never
culled), so vsa-points stays green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
InvertYRenderer captured its owning sink's TextureRenderTarget (color
texture, QRhiTextureRenderTarget and QRhiRenderPassDescriptor) by value
at createRenderer() time and never refreshed it. BackgroundNode::resize()
however destroys the old QRhiTextureRenderTarget / QRhiRenderPassDescriptor
(deleteLater) and installs freshly-created ones. When the resize takes the
in-place fast path (RenderList::resizeSwapchainSizedTargets -> maybeRebuild
release()+init()), the renderer is rebuilt but NOT reconstructed, so its
cached m_inputTarget kept pointing at the now-freed render target and
render-pass descriptor. The upstream node's final pass
(RenderedISFNode::addOutputPass -> renderTargetForOutput ->
renderTargetForInput) then rebuilt its graphics pipeline against that stale
QRhiRenderPassDescriptor.

On Vulkan this is a use-after-free of the backing VkRenderPass: the handle
is deleteLater()-destroyed at the next beginOffscreenFrame, then bound in
vkCreateGraphicsPipelines. Validation reports VK_ERROR_VALIDATION_FAILED_EXT
(-1000011001) / "renderPass is not a valid render pass"; the NVIDIA driver
may dereference the freed handle and SIGSEGV. Whether the freed memory is
still intact is heap-state dependent, hence the intermittent (~9/10) failure
once prior GPU work has churned the allocator. OpenGL has no VkRenderPass
object so it was unaffected.

Fix: add OutputNode::currentRenderTarget() (default empty = no refresh),
override it in BackgroundNode to return its live target, and have
InvertYRenderer::init() re-adopt it. The output renderer is first in
RenderList::renderers, so the refresh runs before the upstream nodes are
re-init'd in the same maybeRebuild pass, and their pipelines bind the live
render-pass descriptor. Also removes the cascade of leaked child objects
(descriptor-set layouts, render pass) observed at device teardown.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
A CSF compute node writing a 3D storage image (EXECUTION_MODEL 3D_IMAGE)
read back all-black on the OpenGL backend while Vulkan was correct.

Root cause is in Qt's QRhi OpenGL backend: setShaderResources binds a 3D
(also cube / array) storage image with glBindImageTexture(..., layered=
GL_FALSE, layer=0), exposing only slice 0 of the volume to the shader. An
imageStore into an image3D therefore writes ONLY z=0; every other slice
stays uninitialised, so the downstream sampler3D reads black. (The barrier
QRhi emits between the compute write and the sampler fetch is correct and
present — verified via a glMemoryBarrier trace — so it is not the cause.
Upstream Qt later fixed the bind to treat 3D/cube/array as layered.) The
2D image path is unaffected because a 2D image has no extra slices.

Fix: dispatchComputeLayered3D() (RhiComputeBarrier) — when a compute pass
writes a 3D storage image on the GL backend, flush QRhi's own binding via
beginExternal(), re-bind each 3D storage image LAYERED using the exact
format QRhi chose (queried back from GL, no format table duplicated),
dispatch, and emit a full memory barrier. It returns false — leaving the
ordinary QRhi dispatch path completely untouched — for every non-OpenGL
backend and whenever the pass has no 3D storage image, so Vulkan and the
2D path are byte-for-byte unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
Three defects in the ISF/RawRaster graphics-storage binding path where the
runtime SRB assignment drifted from libisf's GLSL codegen (isf.cpp), plus a
sampler-index miscount. All three verified against the codegen ground truth.

F1 [crash] Multiview UBO binding ignored uniform_input UBOs. The runtime
placed the multiview UBO at max(firstStorageBinding, ssbo.binding+1,
image.binding+1), never scanning m_storage.ubos. But the codegen assigns
bindings from ONE counter across ssbos, images AND uniform_input UBOs and
puts the multiview UBO one past all of them (isf_emit_graphics_storage
isf.cpp:3406-3449 -> isf_emit_multiview_ubo isf.cpp:3773-3783). A MULTIVIEW>=2
shader with a graphics uniform_input holding the top binding collided the
multiview UBO with the camera UBO and left the shader's real multiview binding
without an SRB descriptor -> Vulkan/D3D12 crash, GL aliasing. Fix: record the
next-free binding in GraphicsStorageResources::nextBinding (exactly
isf_emit_graphics_storage's return value) and reuse it verbatim at all three
multiview sites (RenderedISFNode, SimpleRenderedISFNode initPass + initMRTPass)
so runtime and codegen can never drift.

F2 [correctness] Storage-visibility filter diverged from codegen. The runtime
skipped only stages=={} (or ComputeStage), but visibilityToStages() maps
"all"/unknown to fragment, so those consumed a runtime binding the codegen
(is_graphics_visibility, isf.cpp:3269/3413/3429/3442) skips -> every later
storage binding drifted one slot, silently binding the wrong buffer/image.
Fix: gate collectGraphicsStorageResources on a new isGraphicsVisibility()
predicate that mirrors is_graphics_visibility verbatim. This also corrects the
RawRaster path, which shares the collector.

F3 [correctness] updateInputSamplerFilter miscounted sampler index across
SamplableDepth ports. initInputSamplers pushes TWO samplers for a
SamplableDepth image port (color + depth companion, Utils.cpp:1420-1432), but
updateInputSamplerFilter advanced sampler_idx only once per Image port, unlike
its sibling updateInputTexture. Any port after a SamplableDepth image edited
the wrong QRhiSampler. Fix: add the extra sampler_idx++ for SamplableDepth in
both overloads (RenderedISFNode + SimpleRenderedISFNode).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
resolveDynamicSlot caches a non-owning raw QRhiTexture* per dynamic slot,
keyed by globalResourceId. When a producer (video/NDI/window-capture)
changes resolution it frees the old texture and creates a new one with a
fresh id: resolveDynamicSlot returns a different slot for the new id and the
old slot keeps a dangling pointer. The consumer binds every non-null dynamic
slot as an aux sampler each frame, so the freed texture was dereferenced
during SRB build/draw forever (the 4-slot map rarely fills, so LRU eviction
never fired).

There is no teardown callback from producers, so detect staleness
structurally: resolveDynamicSlot stamps each slot it resolves this frame with
a fresh monotonic counter value; sweepStaleDynamicTextureSlots() clears any
slot not re-resolved since the previous sweep (dangling / orphaned) so it can
no longer be bound. The sweep is hooked into the existing per-frame
sweepMeshSlabs call, which the preprocessor invokes after its resolve pass and
before binding the slots — the exact ordering the sweep needs to never clear a
live slot. Misses now also reuse a previously-cleared slot before appending,
keeping the slot count bounded to the live set.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
In the surgical rt_changed path (Phase B), when an intermediate node's input
render target actually changes shape at runtime (a render_target_spec message,
no restart/resize), the code calls releaseState() then initState(). releaseState
clears the renderer's own per-edge output pass list (m_p / m_passes) and pipeline
cache, but initState deliberately does NOT recreate output passes — only init()
does, via addOutputPass. Phase C then only re-adds the UPSTREAM producers feeding
this node's changed inputs, never this node's OWN output passes. Result: the node
(NodeB in A->B->Output) silently stopped producing into its unchanged downstream
sinks until a full RenderList rebuild.

Rebuild the node's own output passes right after initState, exactly as init()
does (iterate the node's Image output ports, addOutputPass per existing edge).
This is disjoint from the Phase C upstream re-add.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
incrementalEdgeUpdate's addition loop skips any edge whose source/sink node is
not yet present (its ADD_NODE command has not been dequeued — the node and edge
channels are independent). But updateGraph already committed cur_edges to the
authoritative `edges` baseline, so the skipped edge was treated as applied: the
next diff saw old_edges == cur_edges for it and never re-emitted it, losing the
connection until an unrelated full rebuild.

Collect skipped edges whose endpoint node is absent, roll them back out of the
`edges` baseline, and re-raise edges_changed so the next tick — by which the
node has been added — re-diffs and wires them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
removeNodeAndEdges cleaned up only the removed node's own renderers then called
the bare retopologicalSort() per render list. Removing an intermediate node can
make its whole upstream chain unreachable (A->M->N->Output: removing N orphans M
and A). retopologicalSort rebuilds rl->nodes/renderers from the reachable set but
never releases the orphaned upstream renderers (GPU-resource leak) nor erases
their node->renderedNodes[rl] entries — a later use-after-free when the RenderList
is destroyed and those renderers call releaseState on the freed list.

Replace the retopologicalSort loop with reconcileAllRenderLists(), whose step 3
already deletes+erases unreachable renderers (the same cleanup the edge-removal
path relies on) and whose step 8 notifies each output. A removal makes no new
node reachable, so renderer creation (step 5) is a no-op.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
dispatchComputeLayered3D only re-bound storage images flagged
ThreeDimensional, but Qt's GL backend (qrhigles2.cpp, before the fix that
shipped in 6.10) binds cube maps and 2D texture arrays non-layered
identically: `layered = CubeMap || ThreeDimensional || TextureArray`. So a
CSF compute shader writing a write_only/read_write imageCube or
image2DArray storage image only wrote face/layer 0 and read back black on
Qt 6.4.2 GL, exactly like the 3D case.

Broaden the SRB scan predicate to match qrhigles2's `layered` determination
EXACTLY (3D || CubeMap || TextureArray); glBindImageTexture(layered=GL_TRUE,
layer=0) exposes all faces/slices/layers for all three, so the existing
rebind loop covers them. Rename dispatchComputeLayered3D ->
dispatchComputeLayeredImages and update its callers/doc so it no longer
reads as 3D-only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
GPUBufferScatter::dispatch computed workgroups=(element_count+255)/256 and
issued cb.dispatch(workgroups,1,1) with no clamp. A geometry input with
>~16.7M elements needing GPU format conversion produces workgroups>65535,
which is an invalid dispatch (GL_INVALID_VALUE, or
VUID-vkCmdDispatch-groupCountX-00386 / lost device on Vulkan) and yields
collapsed/blank geometry.

Mirror the existing 1D_BUFFER clamp in RenderedCSFNode::runInitialPasses:
compute totalWorkgroups in int64 and spread across the Y (and Z) axes so no
axis exceeds rhi.resourceLimit(MaxThreadGroupsPerDimension) (cached in
init(), default 65535). Because the scatter shader indexed purely by
gl_GlobalInvocationID.x, the shader-side global-index math is also fixed to
reconstruct the linear element index from all three invocation components
using the grid dims; those dims are passed via the params UBO (not
gl_NumWorkGroups, which SPIRV-Cross cannot bake to HLSL) and computed by the
same helper used for the dispatch, keeping dispatch layout and shader index
math consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
HWTransferDecoder::exec() detected a software-format change and immediately
released the delegate's plane QRhiTextures/samplers and rebuilt with new ones,
repointing samplers. But the owning renderer's pipeline SRBs
(DirectVideoNodeRenderer::m_p / VideoNodeRenderer::m_p) were baked with the OLD
sampler.texture pointers and were never rebuilt, so the next render pass sampled
freed textures -> UAF (a regression: the prior code leaked instead).

Defer the teardown: add GPUVideoDecoder::formatChanged, mirroring `failed`.
exec() now records the new format and raises the flag without freeing anything
(old textures stay valid & bound for the rest of the frame). The renderer checks
the flag right after exec() and calls setupGpuDecoder(), which tears down the
decoder and its pipelines together and recreates textures + SRBs in lockstep.
The Direct renderer adopts the new software format (m_hwSwFormat) so the rebuilt
HWTransferDecoder is constructed for it and the change converges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
poll_bus_errors() popped GST_MESSAGE_ERROR | GST_MESSAGE_WARNING and, on ANY
match, set m_started=false — treating a benign WARNING (late/dropped buffers,
missing PTS, encoder rate warnings, all routine during healthy encoding) like a
fatal ERROR. Since stop_pipeline() early-returns on !m_started, this neutralized
EOS finalization, so the muxer never wrote its moov atom / cluster index and the
output file was left truncated/unplayable.

Two fixes: (a) filter on GST_MESSAGE_ERROR alone so warnings are drained but
ignored (GstMessage is opaque here, so the type can't be read post-pop; a
single-type filter discards the non-matching warnings while still catching real
errors). (b) Decouple "stop feeding frames" from "needs EOS finalization": a new
m_feeding flag gates frame pushes, while m_started keeps stop_pipeline() emitting
EOS and driving the pipeline to GST_STATE_NULL even after a genuine error.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
stage() inserted a DecodedAsset with refcount==0 and added its bytes to
m_total_bytes but left in_lru=false, so it never entered the cold LRU. Since
trim()/maybeAutoTrim() evict only from the cold pool, an asset that was staged
(e.g. a decoded glTF baseColor.jpg) but whose material was removed before any
acquire() stayed in the table forever -> unbounded growth over a session.

Seed freshly-staged entries into the cold LRU (in_lru=true, m_cold_bytes +=
byte_size) so an unconsumed stage is trimmable. acquire() already splices a cold
entry back out of the pool and decrements m_cold_bytes on first use, keeping the
byte accounting consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A8dWKnm8kgkuDaMMjapgFT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants